feature centroid
Enhancing Dimension-Reduced Scatter Plots with Class and Feature Centroids
Hier, Daniel B., Obafemi-Ajayi, Tayo, Olbricht, Gayla R., Burns, Devin M., Petrenko, Sasha, Wunsch, Donald C. II
Dimension reduction is increasingly applied to high-dimensional biomedical data to improve its interpretability. When datasets are reduced to two dimensions, each observation is assigned an x and y coordinates and is represented as a point on a scatter plot. A significant challenge lies in interpreting the meaning of the x and y axes due to the complexities inherent in dimension reduction. This study addresses this challenge by using the x and y coordinates derived from dimension reduction to calculate class and feature centroids, which can be overlaid onto the scatter plots. This method connects the low-dimension space to the original high-dimensional space. We illustrate the utility of this approach with data derived from the phenotypes of three neurogenetic diseases and demonstrate how the addition of class and feature centroids increases the interpretability of scatter plots.
Personalized Federated Learning with Feature Alignment and Classifier Collaboration
Xu, Jian, Tong, Xinyi, Huang, Shao-Lun
Data heterogeneity is one of the most challenging issues in federated learning, which motivates a variety of approaches to learn personalized models for participating clients. One such approach in deep neural networks based tasks is employing a shared feature representation and learning a customized classifier head for each client. However, previous works do not utilize the global knowledge during local representation learning and also neglect the fine-grained collaboration between local classifier heads, which limit the model generalization ability. In this work, we conduct explicit local-global feature alignment by leveraging global semantic knowledge for learning a better representation. Moreover, we quantify the benefit of classifier combination for each client as a function of the combining weights and derive an optimization problem for estimating optimal weights. Modern learning tasks are usually enabled by deep neural networks (DNNs), which require huge quantities of training data to achieve satisfied model performance (Lecun et al., 2015; Krizhevsky et al., 2012; Hinton et al., 2012). However, collecting data is too costly due to the increasingly large volume of data or even prohibited due to privacy protection. Hence, developing communicationefficient and privacy-preserving learning algorithms is of significant importance for fully taking advantage of the data in clients, e.g., data silos and mobile devices (Yang et al., 2019; Li et al., 2020a). To this end, federated learning (FL) emerged as an innovative technique for collaborative model training over decentralized clients without gathering the raw data (McMahan et al., 2017). A typical FL setup employs a central server to maintain a global model and allows partial client participation with infrequent model aggregation, e.g., the popular FedAvg, which has shown good performance when local data across clients are independent and identically distributed (IID). However, in the context of FL, data distributions across clients are usually not identical (non-IID or heterogeneity) since different devices generate or collect data separately and may have specific preferences, including feature distribution drift, label distribution skew and concept shift, which make it hard to learn a single global model that applies to all clients (Zhao et al., 2018; Zhu et al., 2021a; Li et al., 2022). To remedy this, personalized federated learning (PFL) has been developed, where the goal is to learn a customized model for each client that has better performance on local data while still benefiting from collaborative training (Kulkarni et al., 2020; Tan et al., 2021a; Kairouz et al., 2021).
Dataset Structural Index: Leveraging a machine's perspective towards visual data
But when it came to visual datasets, the field immediately stepped towards the algorithmic side. One of the fundamental reasons was the amount of information needed to translate from an image. But with the introduction of convolutional networks and transfer learning [1], [2], [3], it is possible to convert an image or a visual object into feature vectors without losing too much information about the entity under concern. It defined a way to use feature maps to compare and distinguish one visual object from another [4]. There has been a lot of work in using these feature vector conversions in systems like content-based image retrievals [5], using feature vectors as representations of different scenarios [6], [7]. It is critical to understand that there is a difference between the way a machine looks at the data and the way we do. There are scenarios in which the interpretation through features is a little different from the interpretation of humans. DSI is there to bridge the gap and understand the machine's perspective before molding it to shape better architectures, in turn, better model performances. I think two concepts could be linked together to understand a machine's viewpoint while working with visual